Incremental Multi-Word Recognition

ABSTRACT

In one example, a computing device includes at least one processor that is operatively coupled to a presence-sensitive display and a gesture module operable by the at least one processor. The gesture module may be operable by the at least one processor to output, for display at the presence-sensitive display, a graphical keyboard comprising a plurality of keys and receive an indication of a continuous gesture detected at the presence-sensitive display, the continuous gesture to select a group of keys of the plurality of keys. The gesture module may be further operable to determine, in response to receiving the indication of the continuous gesture and based at least in part on the group of keys of the plurality of keys, a candidate phrase comprising a group of candidate words.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/703,820, filed Sep. 13, 2017, which is a continuation of U.S.application Ser. No. 15/264,876, filed Sep. 14, 2016, now U.S. Pat. No.9,798,718, issued Oct. 24, 2017, which is a continuation of U.S.application Ser. No. 14/851,759, filed Sep. 11, 2015, now U.S. Pat. No.9,542,385, issued Jan. 10, 2017, which is a continuation of U.S.application Ser. No. 14/196,552, filed Mar. 4, 2014, now U.S. Pat. No.9,134,906, issued Sep. 15, 2015, which is a continuation of U.S.application Ser. No. 13/787,513, filed Mar. 6, 2013, now U.S. Pat. No.8,701,032, issued Apr. 15, 2014, which claims the benefit of U.S.Provisional Application Ser. No. 61/714,696, filed Oct. 16, 2012, theentire contents of each of which are hereby incorporated by reference.

BACKGROUND

Some computing devices (e.g., mobile phones, tablet computers, etc.) mayprovide a graphical keyboard as part of a graphical user interface forcomposing text using a presence-sensitive display (e.g., screen). Thegraphical keyboard may enable a user of the computing device to entertext (e.g., an e-mail, a text message, or a document, etc.). Forinstance, a presence-sensitive display of a computing device may presenta graphical (or “soft”) keyboard that enables the user to enter data byindicating (e.g., by tapping) keys displayed at the presence-sensitivedisplay.

Gesture-based keyboards may be used to input text into a smartphone.Such keyboards may suffer from limitations in accuracy, speed, andinability to adapt to the user. In some examples, a gesture-basedkeyboard may include functionality to provide word predictions and/orautocorrections of character strings entered by a user. As a userbecomes accustomed to a gesture-based keyboard, the user may wish toenter character strings with many characters in a single gesture. Insome examples, prediction and/or autocorrection accuracy may diminish asthe number of characters included in a single gesture increases.

SUMMARY

In one example, a method includes outputting, by a computing device andfor display at a presence-sensitive display operatively coupled to thecomputing device, a graphical keyboard comprising a plurality of keys,and receiving, by the computing device, an indication of a gesturedetected at the presence-sensitive display, a first portion of thegesture to select a first key of the plurality of keys and a secondportion of the gesture to select a second key of the plurality of keys.The method further includes determining, by the computing device andbased at least in part on the first key, a word-level token comprising asingle string of a plurality of predicted characters, and determining,by the computing device, that the word-level token represents acandidate word included in a lexicon. The method further includesdetermining, by the computing device and in response to determining thatthe word-level token represents the candidate word in the lexicon, aphrase-level token based at least in part on the word-level token andthe second character key, wherein the phrase-level token comprises aplurality of character strings.

In another example, a computer-readable storage medium is encoded withinstructions that, when executed, cause at least one processor of acomputing device to output, for display at a presence-sensitive displayoperatively coupled to the computing device, a graphical keyboardcomprising a plurality of keys, and receive an indication of acontinuous gesture detected at the presence-sensitive display, thecontinuous gesture to select a group of keys of the plurality of keys.The computer-readable storage medium is further encoded withinstructions that, when executed, cause the at least one processor todetermine, in response to receiving the indication of the continuousgesture to select the group of keys, a phrase-level token representing aplurality of candidate words, wherein the phrase-level token comprises aplurality of character strings.

In another example, a device includes at least one processor that isoperatively coupled to a presence-sensitive display, and a gesturemodule operable by the at least one processor to output, for display atthe presence-sensitive display, a graphical keyboard comprising aplurality of keys, and receive an indication of a continuous gesturedetected at the presence-sensitive display, the continuous gesture toselect a group of keys of the plurality of keys. The gesture module isfurther operable by the at least one processor to determine, in responseto receiving the indication of the continuous gesture and based at leastin part on the group of keys of the plurality of keys, a candidatephrase comprising a group of candidate words.

The details of one or more examples of the disclosure are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages will be apparent from the description anddrawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example computing device thatmay be used to incrementally determine text from a gesture, inaccordance with one or more techniques of the present disclosure.

FIG. 2 is a block diagram illustrating further details of one example ofa computing device as shown in FIG. 1, in accordance with one or moretechniques of the present disclosure.

FIGS. 3A-C are block diagrams illustrating further details of oneexample of a computing device shown in FIG. 1, in accordance with one ormore techniques of the present disclosure.

FIGS. 4A-B are flow diagrams illustrating example operations of acomputing device to determine a candidate word and/or phrase from agesture, in accordance with one or more techniques of the presentdisclosure.

FIG. 5 is a flow diagram illustrating example operations of a computingdevice to determine a candidate word and/or phrase from a gesture, inaccordance with one or more techniques of the present disclosure.

FIG. 6 is a block diagram illustrating an example computing device thatoutputs graphical content for display at a remote device, in accordancewith one or more techniques of the present disclosure.

DETAILED DESCRIPTION

In general, this disclosure is directed to techniques for incrementallydetermining one or more candidate words of a candidate phrase based on adetected gesture (e.g., a gesture to select a sequence of charactersincluded in a graphical keyboard). In some examples, apresence-sensitive display device that displays a graphical keyboard mayalso detect gestures. The presence-sensitive display (e.g., atouch-sensitive screen) may enable a user to input text by detectinguser inputs in the form of gestures performed at or near thepresence-sensitive display. In certain examples, a user may enter astring of text (for example a word or phrase), by performing one or moregestures at or near the presence-sensitive display. Techniques describedherein may improve a user's ability to enter such text using a graphicalkeyboard.

For instance, rather than performing multiple discrete gestures to inputa single word or phrase (e.g., multiple touch gestures, each discretetouch gesture to select a character of the word or phrase), techniquesof this disclosure may enable a user to perform a single gesture thatindicates the word or phrase. As an example, using techniques of thisdisclosure, a user may perform a continuous motion gesture thatindicates a multi-word phrase without removing an input unit used toprovide the gesture (e.g., a finger, pen, stylus, and the like) from thepresence-sensitive display.

As the user performs the gesture, the computing device may incrementallydetermine a group of keys of the graphical keyboard indicated by thegesture. The incremental determinations may include searching for one ormore points of a gesture that each align with a given keyboard positionof a key that corresponds to a given letter. The search may includeselecting a point of the gesture that best aligns with the letter of thekeyboard.

To determine candidate words indicated by the gesture, the computingdevice may determine one or more word-level tokens, each of whichincludes a string of predicted characters indicated by the gesture. Eachword-level token may be a prefix of one or more words included in alexicon (e.g., dictionary). The computing device may determine one ormore candidate words indicated by the gesture based on the word-leveltoken. To determine candidate phrases, the computing device maydetermine one or more phrase-level tokens, each of which includes aplurality of character strings indicated by the gesture. That is, inaddition to representing a prefix of one or more words included in thelexicon, a word-level token may itself represent a complete wordincluded in the lexicon. A word-level token that represents a candidateword included in the lexicon may indicate that the next key indicated bythe gesture represents the start of a new candidate word. As such, inresponse to determining that a word-level token represents a candidateword in the lexicon, the computing device may determine a phrase-leveltoken that includes multiple character strings. As the user provides thegesture, the computing device may incrementally determine one or morecandidate words and one or more candidate phrases indicated by thegesture.

As such, using techniques described herein, a computing device maydetermine one or more probable interpretations for a gesture based atleast in part on both the gesture and various states in a lexicon inparallel. In this way, techniques disclosed herein may incrementallymatch the gesture to words and phrases in a data structure, such as alexicon trie, one node/letter at a time, using a spatial and/or lexicalgesture model.

By enabling the user to enter a multi-word phrase with a single gestureand performing incremental determinations to identify candidate wordsincluded in the phrase, techniques of this disclosure may enable theuser to increase the rate at which text is entered. Consequently,techniques of the disclosure may relieve a user from performing a tapgesture for each letter of each word in a phrase, which may be difficultfor a user and/or may result in a decreased text-entry rate due to therequirement that the user's finger discretely contact individual keys.Moreover, the user can, with a single gesture, select a number of keysthat correspond to characters of multiple words in a phrase. In thisway, the user can continuously swipe to select numerous characters, andtechniques of the disclosure may automatically and incrementallydetermine whether the characters correspond to a single word or a phrasethat includes a group of words. The techniques may also reduce a user'seffort to accurately contact individual keys.

FIG. 1 is a block diagram illustrating an example computing device 2that may be used to incrementally determine text from a gesture, inaccordance with one or more techniques of the present disclosure. Insome examples, computing device 2 may be associated with user 18. A userassociated with a computing device may interact with the computingdevice by providing various user inputs into the computing device.

Examples of computing device 2 may include, but are not limited to,portable or mobile devices such as mobile phones (including smartphones), laptop computers, desktop computers, tablet computers, smarttelevision platforms, cameras, personal digital assistants (PDAs),servers, mainframes, etc. As shown in the example of FIG. 1, computingdevice 2 may be a tablet computer. Computing device 2, in some examples,may include user interface (UI) device 4, UI module 6, gesture module 8,and language model 10. Other examples of computing device 2 thatimplement techniques of this disclosure may include additionalcomponents not shown in FIG. 1.

Computing device 2 may include UI device 4. In some examples, UI device4 may be configured to receive tactile, audio, or visual input. UIdevice 4, as shown in FIG. 1, may include a touch-sensitive and/orpresence-sensitive display or any other type of device for receivinginput. UI device 4 may output content such as graphical user interface(GUI) 12 for display. In the example of FIG. 1, UI device 4 may be apresence-sensitive display that may display a graphical user interfaceand receive input from user 18 using capacitive, inductive, and/oroptical detection at or near the presence-sensitive display.

As shown in FIG. 1, computing device 2 may include UI module 6. UImodule 6 may perform one or more functions to receive input, such asuser input, and send such input to other components associated withcomputing device 2, such as gesture module 8. For example, UI module 6may determine a gesture performed by user 18 at UI device 4. UI module 6may also receive data from components associated with computing device2, such as gesture module 8. Using the data, UI module 6 may cause othercomponents associated with computing device 2, such as UI device 4, toprovide output based on the data. For instance, UI module 6 may receivedata from gesture module 8 that causes UI device 4 to displayinformation at committed-text region 14 of GUI 12.

UI module 6 may be implemented in various ways. For example, UI module 6may be implemented as a downloadable or pre-installed application or“app.” In another example, UI module 6 may be implemented as part of ahardware unit of computing device 2. In another example, UI module 6 maybe implemented as part of an operating system of computing device 2.

Computing device 2, in some examples, includes gesture module 8. Gesturemodule 8 may include functionality to perform a variety of operations oncomputing device 2, such as functionality to incrementally determinetext from a gesture in accordance with the techniques described herein.Gesture module 8 may be implemented in various ways. For example,gesture module 8 may be implemented as a downloadable or pre-installedapplication or “app.” In another example, gesture module 8 may beimplemented as part of a hardware unit of computing device 2. In anotherexample, gesture module 8 may be implemented as part of an operatingsystem of computing device 2.

Gesture module 8 may receive data from components associated withcomputing device 2, such as UI module 6. For instance, gesture module 8may receive gesture data from UI module 6 that causes gesture module 8to determine text from the gesture data. In some examples, gesturemodule 8 determines one or more locations of UI device 4 that aretouched or otherwise detected in response to a user gesture, based oninformation received from UI module 6. In some examples, gesture module8 can determine one or more features associated with a gesture, such asthe Euclidean distance between two alignment points, the length of agesture path, the direction of a gesture, the curvature of a gesturepath, the shape of the gesture, and maximum curvature of a gesturebetween alignment points, speed of the gesture, etc. Gesture module 8may also send data to components associated with computing device 2,such as UI module 6. For instance, gesture module 8 may send textdetermined from the gesture data to UI module 6 that causes UI device 4to display GUI 12.

As shown in FIG. 1, GUI 12 may be a user interface generated by UImodule 6 that allows user 18 to interact with computing device 2. GUI 12may include graphical content. Graphical content, generally, may includetext, images, a group of moving images, etc. As shown in FIG. 1,graphical content may include graphical keyboard 16, committed-textregion 14, and text-suggestion regions 24A-C(collectively“text-suggestion areas 24”). Graphical keyboard 16 may include aplurality of keys, such as “Q” key 20A, “W” key 20B, “E” key 20C, “R”key 20D, “A” key 20E, “S” key 20F, “I” key 20G, “O” key 20H, “M” key20J, and “N” key 20K. In some examples, each of the plurality of keysincluded in graphical keyboard 16 represents a single character. Inother examples, one or more of the plurality of keys included ingraphical keyboard 16 represents a group of characters selected based ona plurality of modes.

In some examples, committed-text region 14 may include characters orother graphical content that are included in, for example, atext-message, a document, an e-mail message, a web browser, and thelike. For instance, committed-text region 14 may include characters orother graphical content that are selected by user 18 via gesturesperformed at UI device 4. In some examples, text-suggestion regions 24may each display a word and/or multi-word phrase. As illustrated in theexample of FIG. 1, text-suggestion regions 24 may be different regionsof GUI 12 than committed-text region 14. In other examples,text-suggestion regions 24 may be a single region of GUI 12, and mayinclude one or more regions of GUI 12 that are the same ascommitted-text region 14. Similarly, while illustrated as separateregions in the example of FIG. 1, text-suggestion regions 24, in someexamples, may be a single region of GUI 12.

UI module 6 may cause UI device 4 to display graphical keyboard 16 anddetect a gesture having gesture path 22 which is incrementallydetermined by gesture module 8 in accordance with techniques describedherein. Additionally, UI module 6 may cause UI device 4 to display acandidate word and/or phrase determined from the gesture in one or moreof text-suggestion regions 24.

Graphical keyboard 16 may be displayed by UI device 4 as an ordered setof selectable keys. Keys may represent a single character from acharacter set (e.g., letters of the English alphabet), or may representcombinations of characters. One example of a graphical keyboard mayinclude a traditional “QWERTY” keyboard layout. Other examples maycontain characters for different languages, different character sets, ordifferent character layouts. As shown in the example of FIG. 1,graphical keyboard 16 includes a version of the traditional “QWERTY”keyboard layout for the English language providing character keys aswell as various keys (e.g., the “?123” key) providing otherfunctionality.

Computing device 2, in some examples, includes language model 10.Language model 10 may include a lexicon. In some examples, a lexicon mayinclude a listing of words and may include additional information aboutthe listed words. A lexicon may be represented by one or more datastructures, such as by one or more of an array, a list, a tree, or otherdata structures. For example, language model 10 may include a lexiconstored in a trie data structure. A lexicon trie data structure mayinclude a plurality of nodes. Each node of the lexicon trie mayrepresent a letter. The first node in a lexicon trie may be consideredan entry node, which may not correspond to a letter. In other examples,the entry node may correspond to a letter. Each node may have one ormore child nodes. For instance, the entry node may have twenty-six childnodes, each corresponding to a letter of the English alphabet.

A subset of the nodes in a lexicon trie may each include a flag whichindicates that the node is a terminal node. Each terminal node of alexicon trie may indicate a complete word (e.g., a candidate word)included in the lexicon. The letters indicated by the nodes along a pathof nodes from the entry node to a terminal node may spell out a wordindicated by the terminal node. In some examples, language model 10 maybe a default dictionary installed on computing device 2. In certainexamples, language model 10 may include a group of predefined phrasesinstalled on computing device 2. In other examples, language model 10may include multiple sources of lexicons, which may be stored atcomputing device 2 or stored at one or more remote computing devicesthat are accessible to computing device 2 via one or more communicationchannels.

In some examples, language model 10 may be implemented in the firmwareof computing device 2. Language model 10 may include language modelfrequency information such as n-gram language models. An n-gram languagemodel may provide a probability distribution for an item x_(i) (letter,word, punctuation character or other delimiter) in a contiguous sequenceof items based on the previous items in the sequence (i.e.,P(x_(i)|_(xi-(n-1)), . . . , x_(i-1))). For instance, a bigram languagemodel (an n-gram model where n=2), may provide a probability that theletter “w” follows the sequence of letters “no”. As another example, atrigram language model (an n-gram model where n=3) may provide aprobability that the word “to” follows the sequence of words “we aim”.In certain examples, a trigram language model may provide a probabilitythat a delimiter character (e.g., a comma delimiter character, a perioddelimiter character, a semicolon delimiter character) is positionedbetween a first character string and a second character string. Forinstance, a trigram language model may provide a probability that acomma delimiter character is positioned between a first character string“example” and a second character string “the.” In some examples,language model 10 includes a lexicon trie with integrated language modelfrequency information. For instance, each node of the lexicon trie mayinclude a representation of a letter and a probability value.

Techniques of the present disclosure may improve the speed and accuracywith which a user can enter text into a computing device. Usingtechniques of this disclosure, a user may, instead of performing adiscrete gesture for each key of a word, perform a single gesture thatindicates the word. Similarly, according to techniques described herein,a user may perform a single gesture that indicates characters ofmultiple words (e.g., two words, three words, five words, or othernumbers of words) in a phrase.

As the user performs the gesture, the computing device may incrementallydetermine a group of keys indicated by the gesture. Each key may beassociated with one or more characters. The computing device maydetermine, based on the determined group of keys, one or more word-leveltokens, each of the word-level tokens including a single string of aplurality of predicted characters. The computing device may determineone or more candidate words indicated by the gesture based on theword-level tokens. In addition, the computing device may determine thata word-level token itself represents a complete word included in alexicon. In response, the computing device may determine a phrase-leveltoken that includes a plurality of character strings. For instance, thecomputing device may determine the phrase-level token as a combinationof the word-level token that represents the complete word included inthe lexicon and another word-level token that begins with a nextselected key indicated by the gesture. By incrementally decoding thegesture as it is being performed, the user may be presented with acandidate word and/or phrase with minimal post-gesture entry processingtime. Moreover, by enabling the user to enter a word and/or phrase witha single gesture, techniques of this disclosure may enable the user toincrease the rate at which text is entered.

As shown in the example of FIG. 1, UI module 6 may output GUI 12including graphical keyboard 16 for display at UI device 4. UI device 4(e.g., a presence-sensitive and/or touch-sensitive display) may detect agesture to select one or more keys of graphical keyboard 16. In oneexample, the gesture may be a continuous motion gesture that includes amotion of an input unit (e.g., a finger, pen, stylus, etc.) from a firstlocation of UI device 4 to a second location of UI device 4 such thatthe gesture performed from the first location to the second location isdetected by UI device 4 throughout the performance of the gesture. Forinstance, such as when UI device 4 includes a touch-sensitive display,the gesture may include a motion of an input unit from the firstlocation to the second location with substantially constant contactbetween the input unit and UI device 4. As illustrated in the example ofFIG. 1, UI device 4 may detect a gesture including gesture path 22provided by a finger of user 18 from a location of UI device 4 thatdisplays “W” key 20B to a location of UI device 4 that displays “M” key20J along gesture path 22 such that UI device 4 detects the fingerthroughout gesture path 22.

The gesture may include a plurality of portions. In some examples, thegesture may be divided into portions with substantially equivalent timedurations. Where the gesture includes a plurality of portions, thegesture may include a final portion which may be a portion of thegesture detected prior to detecting that the gesture is complete. Forinstance, a portion of the gesture may be designated as the finalportion where user 18 moves his/her finger out of proximity with UIdevice 4 such that the finger is no longer detected by UI device 4.

As illustrated, user 18 may perform a gesture to select a group of keysof the plurality of keys. In the example of FIG. 1, UI module 6 mayincrementally detect the gesture having gesture path 22 at thepresence-sensitive display as user 18 performs the gesture by tracinggesture path 22 through or near keys of keyboard 16 that correspond tothe characters of a desired phrase (e.g., the characters correspondingto the phrase “we aim”, represented by “W” key 20B, “E” key 20C, “A” key20E, “I” key 20G, and “M” key 20J). UI module 6 may send data thatindicates gesture path 22 to gesture module 8. In some examples, UImodule 6 incrementally sends data indicating gesture path 22 to gesturemodule 8 as gesture path 22 is detected by UI device 4 and received byUI module 6. For instance, UI module 6 may send a stream of coordinatepairs indicating gesture path 22 to gesture module 8 as gesture path 22is detected by UI device 4 and received by UI module 6. As in theexample of FIG. 1, gesture module 8 may receive an indication of gesturepath 22 from UI module 6, including portion 22A of gesture path 22 toselect “E” key 20C and portion 22B of gesture path 22 to select “A” key20E.

In response to receiving data that represents gesture path 22, gesturemodule 8 may determine one or more word-level tokens, each of theword-level tokens including a single string of a plurality of predictedcharacters. For example, based at least in part on the indication ofportion 22A of gesture path 22, gesture module 8 may determine one ormore word-level tokens, each of the word-level tokens including a singlestring of predicted characters indicated by portion 22A. As an example,gesture module 8 may determine a first word-level token as the string ofpredicted characters “qe” corresponding to an indication of a predictedselection of “Q” key 20A and “E” key 20C. Similarly, gesture module 8may determine a second word-level token as the string of predictedcharacters “we” corresponding to an indication of a predicted selectionof “W” key 20B and “E” key 20C. Gesture module 8 may incrementallydetermine multiple such word-level tokens based at least in part on oneor more selected keys of graphical keyboard 16. Each character of eachword-level token may be associated with a region of UI device 4 thatdisplays a key corresponding to the character. Gesture module 8 maydetermine the one or more word-level tokens based on observed touchpoints relative to the area of UI device 4 that displays the one or morekeys corresponding to the one or more characters of the word-leveltoken.

Each of the word-level tokens including the single string of predictedcharacters may be a prefix of a word included in the lexicon. Gesturemodule 8 may determine one or more candidate words based at least inpart on the one or more word-level tokens. A candidate word may be aword suggested to the user that is composed of a group of keys indicatedby gesture path 22. As an example, as described above, gesture module 8may determine one or more word-level tokens in response to receiving anindication of portion 22A of gesture path 22, such as a first word-leveltoken including the string of predicted characters “qe”, a secondword-level token including the string of predicted characters “we”, athird word-level token including the string of predicted characters“wr”, or other word-level tokens. One or more of the word-level tokensmay be a prefix of a word included in a lexicon. Gesture module 8 may,in certain examples, incrementally determine one or more candidate wordsas one or more of the words included in the lexicon for which aword-level token is a prefix.

Additionally, one or more of the word-level tokens may represent acandidate word included in the lexicon. For instance, in the presentexample, gesture module 8 may determine that the second word-level tokenincluding the string of predicted characters “we”, in addition to beinga prefix of words included in the lexicon (e.g., the words “went”,“were”, etc.), itself represents a candidate word included in thelexicon (i.e., the word “we” in the English language). As such, gesturemodule 8 may determine that a next selected key indicated by gesturepath 22, rather than representing a next letter of a word for which thestring of characters “we” is a prefix, may represent a first letter of anext word. In response to determining that the word-level tokenrepresents the candidate word in the lexicon, gesture module 8 maydetermine a phrase-level token based at least in part on the word-leveltoken and a next predicted character key as indicated by gesture path22.

For example, as illustrated in the example of FIG. 1, UI module 6 mayreceive an indication of portion 22B of gesture path 22 detected at UIdevice 4. Gesture module 8 may receive data from UI module 6corresponding to the indication of portion 22B of gesture path 22. Inresponse to determining that the word-level token including the singlestring of predicted characters “we” represents a candidate word includedin the lexicon, gesture module 8 may generate a phrase-level token basedat least in part on a next selected character as indicated by portion22B of gesture path 22. For instance, gesture module 8 may determine aphrase-level token including the plurality of character strings “we” and“a” separated by a space character. That is, the phrase-level token mayinclude the word-level token that represents the candidate word in thedictionary and a next word-level token that begins with the nextpredicted character as indicated by portion 22B of gesture path 22.

Gesture module 8 may determine candidate phrases based on thephrase-level token, such as by determining a candidate phrase as acombination of the word-level token that represents the candidate wordin the dictionary and a candidate word for which the next word-leveltoken that begins with the next predicted character is a prefix. As anexample, gesture module 8 may determine one or more candidate phrases,such as the phrase “we are”, the phrase “we ask”, and the like. In someexamples, gesture module 8 may cause UI module 6 to output one or moreof the candidate phrases for display at text-suggestion regions 24.

In addition to determining the phrase-level token based at least in parton the indication of the next selected character, gesture module 8 maydetermine a word-level token based at least in part on the next selectedcharacter. For example, gesture module 8 may determine a phrase-leveltoken based at least in part on the word-level token including theplurality of character strings “we” and “a” separated by a spacecharacter, and may determine a word-level token that includes the singlestring of predicted characters “wea”. That is, gesture module 8 mayincrementally determine predicted words and predicted phrases indicatedby gesture path 22 in parallel, thereby enabling a user to provide onecontinuous gesture to select a group of keys included in both candidatewords and candidate phrases.

Gesture module 8 may determine the one or more word-level tokens andphrase-level tokens by determining a group of alignment points traversedby gesture path 22, determining respective cost values for each of atleast two keys of the plurality of keys, and comparing the respectivecost values for at least each of at least two keys of the plurality ofkeys, as further described below.

An alignment point is a point along gesture path 22 that may indicate akey of the plurality of keys included in graphical keyboard 16. Analignment point may include one or more coordinates corresponding to thedetermined position of the alignment point. For instance, an alignmentpoint may include Cartesian coordinates corresponding to a point on GUI12.

In some examples, gesture module 8 determines the group of alignmentpoints traversed by gesture path 22 based on a plurality of featuresassociated with gesture path 22. The plurality of features associatedwith gesture path 22 may include a length of a segment of gesture path22. For instance, gesture module 8 may determine the length along thegesture segment from a previous alignment point and the currentalignment point. For better alignments, the length will more closelyapproximate the straight-line distance between to two correspondingkeyboard letters.

In another example, gesture module 8 may determine a direction of asegment from a first point to a second point of gesture path 22 todetermine the group of alignment points. For better alignments, thedirection of the segment will more closely approximate the direction ofa straight line from between two corresponding keyboard letters.

In some examples, gesture module 8 may determine features of gesturepath 22 such as a curvature of a segment of gesture path 22, a localspeed representing a rate at which a segment of path 22 was detected,and a global speed representing a rate at which gesture path 22 wasdetected. If gesture module 8 determines a slower speed or pause for thelocal speed, gesture module 8 may determine that a point at the segmentis more likely to be an alignment point. If gesture module 8 determinesthat a gesture was drawn quickly, gesture module 8 may determine thatthe gesture is more likely to be imprecise and therefore gesture module8 may apply a greater weight on the language module (i.e., n-gramfrequencies) than the spatial model. In one example, gesture module 8may determine an alignment point of the group of alignment points basedon a segment of gesture path 22 having a high curvature value.Additionally, gesture module 8 may determine an alignment point of thegroup of alignment points based on a segment of gesture path 22 having alow local speed (i.e., the user's finger slowed down while performingthe segment of the gesture). In the example of FIG. 1, gesture module 8may determine a first alignment point at the start of gesture path 22, asecond alignment point at the point where gesture path 22 experiences asignificant change in curvature, and a third alignment point at the endof gesture path 22. In still other examples, techniques of thedisclosure may identify a shape of the gesture as a feature anddetermine an alignment point based on the shape of the gesture.

In some examples, gesture module 8 may determine respective cost valuesfor each of at least two keys of the plurality of keys included inkeyboard 16. Each of the respective cost values may represent aprobability that an alignment point indicates a key. In some examples,the respective cost values may be based on physical features of thegesture path, the alignment point, and/or the key. For instance, therespective cost values may be based on the physical location of thealignment point with reference to the physical location of the key.

In some examples, the respective cost values may be based on languagemodel 10. For instance, the respective cost values may be based on theprobability that a second key will be selected after a first key (e.g.,the probability that the “e” key will be selected after the “w” key). Asanother example, the respective cost values may be based on theprobability that a second candidate word will follow a first candidateword (e.g., the probability that the candidate word “aim” will followthe candidate word “we”). In certain examples, the keys for whichrespective cost values are determined are selected based at least inpart on language model 10. In some examples, the cost values are lowerwhere there is a greater likelihood that an alignment point indicates akey. In other examples, the cost values are higher where there is agreater likelihood that an alignment point indicates a key.

In certain examples, language model 10 may include frequency informationregarding a probability that one or more delimiter characters arepositioned between a first character string and a second characterstring of a phrase-level token. For instance, a phrase-level token mayinclude a first character string including the characters “John” and asecond character string including the characters “this”. In such anexample, language model 10 (e.g., an n-gram language model) may includefrequency information indicating a probability that one or moredelimiter characters are positioned between the first character string(e.g., the character string “John”) and the second character string(e.g., the character string “this”).

For instance, language model 10 may include frequency informationindicating a probability that a comma delimiter is positioned betweenthe first character string “John” and the second character string“this”. In some examples, computing device 2 may determine aphrase-level token as including one or more delimiter characterspositioned between the first character string and the second characterstring based at least in part on the probability indicated by languagemodel 10 that the delimiter character is positioned between the firstand second character strings of the phrase-level token. As such, incertain examples, computing device 2 may automatically insert one ormore delimiter characters in a candidate phrase, thereby enabling a userto provide a single continuous gesture to select one or more charactersof a phrase without providing an input to affirmatively select the oneor more delimiter characters. As such, in certain examples, graphicalkeyboard 16 may not include keys associated with one or more delimitercharacters. For instance, in some examples, graphical keyboard 16 maynot include keys associated with one or more of a space delimitercharacter, a period character, a comma character, a hyphen character, anapostrophe character, or other delimiter characters.

Examples of such delimiter characters may include, but are not limitedto, one or more of a comma delimiter character, a period delimitercharacter, a semicolon delimiter character, a question mark delimitercharacter, a hyphen delimiter character, an apostrophe delimitercharacter, or other punctuation delimiter characters. In general, adelimiter character may include any character (e.g., punctuationcharacter or otherwise) that may be used to separate or otherwisedelimit characters or strings of characters. As an example, in theexample of FIG. 1, language model 10 may include a probability that acomma delimiter character is positioned between the character string“We” and the character string “aim”. As another example, language model10 may include a probability that a period delimiter character ispositioned between the character string “West” and the character string“on”.

In some examples, language model 10 may include a probability that morethan one delimiter character is positioned between the first and secondcharacter strings. For instance, a first character string may includethe characters “e.g” and a second character string may include thecharacters “the”. In such an example, language model 10 may include aprobability that both a period delimiter character and a comma delimitercharacter are positioned between the first and second character strings,such that the phrase-level token includes the phrase “e.g., the” (i.e.,the first character string “i.e” followed by a period delimitercharacter and a comma delimiter character). As another example, languagemodel 10 may include a probability that multiple delimiter characters,such as a period delimiter character and two space delimiter charactersare positioned between the first and second character strings. Forinstance, a first character string may include the characters “end” anda second character string may include the characters “next”. In such anexample, language model 10 may include a probability that a perioddelimiter character and two space delimiter characters are positionedbetween the first and second character strings, such that thephrase-level token includes the phrase “end. Next”.

Gesture module 8 may, in certain examples, insert one or more delimitercharacters between a first character string of a phrase-level token anda second character string of the phrase-level token. For example,gesture module 8 may compare the probability that the one or moredelimiter characters are positioned between the first and secondcharacter strings to a threshold value (e.g., a threshold probability,such as sixty percent, seventy percent, ninety percent, or otherprobabilities). Gesture module 8 may insert the one or more delimitercharacters between the first and second character strings when thedetermined probability satisfies the threshold value, such as when thedetermined probability is greater than (or equal to) the thresholdvalue.

In some examples, gesture module 8 may determine a first phrase-leveltoken that does not include the one or more delimiter characters and asecond phrase-level token that includes the one or more delimitercharacters. In such examples, gesture module 8 may determine respectivecost values for each of the characters included the first phrase-leveltoken that does not include the one or more delimiter characters.Similarly, gesture module 8 may determine respective cost values foreach of the characters included in the second phrase-level token thatincludes the one or more delimiter characters. Gesture module 8 maydetermine one or more candidate phrases based at least in part on therespective cost values for the first phrase-level token and the secondphrase-level token. In such a way, gesture module 8 may incrementallydetermine one or more candidate phrases indicated by a gesture, the oneor more candidate phrases including one or more delimiter characterspositioned between character strings of the candidate phrase.

In the example of FIG. 1, gesture module 8 may determine a first costvalue representing a probability that the first alignment pointindicates “Q” key 20A and a second cost value representing a probabilitythat the first alignment point indicates “W” key 20B. Similarly, gesturemodule 8 may determine a third cost value representing a probabilitythat the second alignment point indicates “E” key 20C and a fourth costvalue representing a probability that the second alignment pointindicates “R” key 20D. In this way, gesture module 8 may incrementallydetermine cost values, each representing a probability that an alignmentpoint indicates a key of the plurality of keys included in graphicalkeyboard 16. For instance, gesture module 8 may determine cost valuesrepresenting probabilities that alignment points indicate “A” key 20E,“S” key 20F, “I” key 20G, “O” key 20H, “M” key 20J, “N” key 20K, orother keys included in the plurality of keys.

Gesture module 8 may compare the respective cost values for at least twokeys of the plurality of keys to determine a combination of keys havinga combined cost value that satisfies a threshold. A combined cost valuemay represent a probability that gesture path 22 indicates a combinationof keys. Gesture module 8 may compare the respective cost values for atleast two keys of the plurality of keys to determine which of the atleast two keys is indicated by an alignment point. Gesture module 8 maydetermine a combination of keys by determining which keys are indicatedby each alignment point. In some examples, gesture module 8 determineswhich of the at least two keys is indicated by an alignment pointwithout regard to which keys are indicated by other alignment points. Inother examples, gesture module 8 determines which of the at least twokeys is indicated by the alignment point based on which keys areindicated by other alignment points. In such examples, gesture module 8may revise the determination of which key is indicated by a previousalignment point based on the respective cost values for a currentalignment point.

In some examples, gesture module 8 may compare the combined cost valueof a determined combination of keys with a threshold value. In someexamples, the threshold value is the combined cost value of a differentdetermined combination of keys. For instance, gesture module 8 maydetermine a first combination of keys having a first combined cost valueand a second combination of keys having a second combined cost value. Insuch an instance, gesture module 8 may determine that a candidate wordor phrase is based on the combination of keys with the lower combinedcost value. In the example of FIG. 1, gesture module 8 may compare thedetermined respective cost values to determine a combination of keys(i.e., “W”, “E”, “A”, “I”, and “M”) having a combined cost value.

In some examples, gesture module 8 begins to determine a candidate wordand/or phrase prior to the time in which UI device 4 completes detectinggesture path 22. In the example of FIG. 1, rather than determining thecandidate word and/or phrase after UI device 4 completes detectinggesture path 22, gesture module 8 may determine a plurality of wordsand/or phrases as gesture path 22 is detected, such as “we”, “went”,“wear”, “we are” “we aim”, and “we aimed”. Additionally, in the exampleof FIG. 1, gesture module 8 may contemporaneously revise the determinedplurality of words as gesture path 22 is detected, such as revision of“we” to “wear”. Furthermore, rather than determining the candidate wordand/or phrase based on a geometric shape of the gesture, techniques ofthe disclosure may determine a candidate word and/or phrase based on agroup of characters indicated by the gesture. Gesture module 8 may sendthe determined word and/or phrase to UI module 6 which may then cause UIdevice 4 to display the word and/or phrase (e.g., the phrase “we aim”)in committed-text region 14 of GUI 12.

In some alternative embodiments, a user can provide an indication that acharacter associated with a key should be included in the candidate wordand/or phrase, such as by pausing for a threshold time duration at a keyto indicate that the character should be included in the candidate wordand/or phrase. Similarly, a user may provide an indication that a nextselected key should be included in a next word of a candidate phrase,such as by pausing for a threshold time duration at or near the lastletter of a word prior to extending the gesture to the first letter ofthe next word, or performing a motion gesture at or near the last letterof a word prior to extending the gesture to the first letter of the nextword (e.g., a looping gesture, an upward motion gesture, a downwardmotion gesture, a motion gesture in a direction toward a space key,etc.). In another alternative embodiment, rather than using a trie basedsearch as described using techniques of the disclosure, gesture module 8may maintain a separate gesture-specific word list or dictionary.

In some examples, techniques of the disclosure provide for efficientperformance on computing devices, for instance, recognizing gestures infewer than 100 milliseconds in some cases. Techniques of the disclosuremay also use the default dictionary installed on the mobile devicerather than using a dedicated gesture dictionary that may be maintainedseparately and use additional storage resources. In this way, techniquesof the disclosure may reduce storage requirements by using a dictionarythat is already stored by a default input entry system. Moreover, thedictionary may be implemented efficiently as a compact lexicon trie.Using a default dictionary already provided on a computing device alsoprovides ready support foreign languages, contact names, and user addedwords in accordance with techniques of the disclosure. By using, e.g., alexicon trie and the default dictionary, techniques of the disclosuremay integrate the language model frequencies (i.e., n-gramprobabilities) into the gesture interpretation, thereby allowing thesearch techniques to concentrate on the most promising paths forcandidate words based on both the shape of the gesture and theprobability of the word being considered.

Additionally, by incrementally determining the selected keys of theplurality of keys indicated by the gesture using both a spatial modeland a language model, techniques described herein may enable thecomputing device to determine a candidate word and/or phrase that is notincluded in the lexicon. For instance, based on a combination of thespatial model including a determination of alignment points traversed bythe gesture and other features of the gesture (e.g., a local speed ofone or portions of the gesture, a global speed of the gesture, acurvature of one or more portions of the gesture, etc.) and the languagemodel (e.g., n-gram frequencies of predicted characters and/or words),the computing device may determine that a most probable group of keysindicated by the gesture includes a group of keys that does not spell aword included in the lexicon. As such, according to techniques describedherein, the computing device may enable a user to provide a singlegesture to select a group of keys that does spell a word included in thelexicon, such as an abbreviation or other such character string.

FIG. 2 is a block diagram illustrating further details of one example ofa computing device shown in FIG. 1, in accordance with one or moretechniques of the present disclosure. FIG. 2 illustrates only oneparticular example of computing device 2 as shown in FIG. 1, and manyother examples of computing device 2 may be used in other instances.

As shown in the specific example of FIG. 2, computing device 2 includesone or more processors 40, one or more input devices 42, one or morecommunication units 44, one or more output devices 46, one or morestorage devices 48, and user interface (UI) device 4. Computing device2, in one example further includes UI module 6, gesture module 8, andoperating system 58 that are executable by computing device 2 (e.g., byone or more processors 40). Computing device 2, in one example, furtherincludes language model 10, key regions 52, active beam 60, and nextbeam 56.

Each of components 4, 40, 42, 44, 46, and 48 may be interconnected(physically, communicatively, and/or operatively) for inter-componentcommunications. In some examples, communication channels 50 may includea system bus, a network connection, an inter-process communication datastructure, or any other method for communicating data. As one example inFIG. 2, components 4, 40, 42, 44, 46, and 48 may be coupled by one ormore communication channels 50. UI module 6 and gesture module 8 mayalso communicate information with one another as well as with othercomponents in computing device 2, such as language model 10, key regions52, active beam 54, and next beam 56.

Processors 40, in one example, are configured to implement functionalityand/or process instructions for execution within computing device 2. Forexample, processors 40 may be capable of processing instructions storedin storage device 48. Examples of processors 40 may include any one ormore of a microprocessor, a controller, a digital signal processor(DSP), an application specific integrated circuit (ASIC), afield-programmable gate array (FPGA), or equivalent discrete orintegrated logic circuitry.

One or more storage devices 48 may be configured to store informationwithin computing device 2 during operation. Storage device 48, in someexamples, is described as a computer-readable storage medium. In someexamples, storage device 48 is a temporary memory, meaning that aprimary purpose of storage device 48 is not long-term storage. Storagedevice 48, in some examples, is described as a volatile memory, meaningthat storage device 48 does not maintain stored contents when thecomputer is turned off. Examples of volatile memories include randomaccess memories (RAM), dynamic random access memories (DRAM), staticrandom access memories (SRAM), and other forms of volatile memoriesknown in the art. In some examples, storage device 48 is used to storeprogram instructions for execution by processors 40. Storage device 48,in one example, is used by software or applications running on computingdevice 2 (e.g., gesture module 8) to temporarily store informationduring program execution.

Storage devices 48, in some examples, also include one or morecomputer-readable storage media. Storage devices 48 may be configured tostore larger amounts of information than volatile memory. Storagedevices 48 may further be configured for long-term storage ofinformation. In some examples, storage devices 48 include non-volatilestorage elements. Examples of such non-volatile storage elements includemagnetic hard discs, optical discs, floppy discs, flash memories, orforms of electrically programmable memories (EPROM) or electricallyerasable and programmable (EEPROM) memories.

Computing device 2, in some examples, also includes one or morecommunication units 44. Computing device 2, in one example, utilizescommunication unit 44 to communicate with external devices via one ormore networks, such as one or more wireless networks. Communication unit44 may be a network interface card, such as an Ethernet card, an opticaltransceiver, a radio frequency transceiver, or any other type of devicethat can send and receive information. Other examples of such networkinterfaces may include Bluetooth, 3G and WiFi radios computing devicesas well as Universal Serial Bus (USB). In some examples, computingdevice 2 utilizes communication unit 44 to wirelessly communicate withan external device such as a server.

Computing device 2, in one example, also includes one or more inputdevices 42. Input device 42, in some examples, is configured to receiveinput from a user through tactile, audio, or video feedback. Examples ofinput device 42 include a presence-sensitive display, a mouse, akeyboard, a voice responsive system, video camera, microphone or anyother type of device for detecting a command from a user. In someexamples, a presence-sensitive display includes a touch-sensitivescreen.

One or more output devices 46 may also be included in computing device2. Output device 46, in some examples, is configured to provide outputto a user using tactile, audio, or video stimuli. Output device 46, inone example, includes a presence-sensitive display, a sound card, avideo graphics adapter card, or any other type of device for convertinga signal into an appropriate form understandable to humans or machines.Additional examples of output device 46 include a speaker, a cathode raytube (CRT) monitor, a liquid crystal display (LCD), or any other type ofdevice that can generate intelligible output to a user.

In some examples, UI device 4 may include functionality of input device42 and/or output device 46. In one example, UI device 4 may be atouch-sensitive screen. In the example of FIG. 2, UI device 4 may be apresence-sensitive display. In some examples, a presence sensitivedisplay may detect an object, such as an input unit (e.g., user'sfinger, stylus, etc.) at and/or near the screen of thepresence-sensitive display. As one example range, a presence-sensitivedisplay may detect a user's finger that is within 2 inches or less ofthe physical screen of the presence-sensitive display. Thepresence-sensitive display may determine one or more locations (e.g.,(x,y) coordinates) of the presence-sensitive display at which the fingerwas detected. In another example range, a presence-sensitive display maydetect an object 6 inches or less from the physical screen of thepresence-sensitive display and other exemplary ranges are also possible.The presence-sensitive display may determine the location of the displayselected by a user's finger using capacitive, inductive, and/or opticalrecognition techniques. In some examples, the presence sensitive displayprovides output to a user using tactile, audio, or video stimuli asdescribed with respect to output device 46.

Computing device 2 may include operating system 58. Operating system 58,in some examples, controls the operation of components of computingdevice 2. For example, operating system 58, in one example, facilitatesthe communication of UI module 6 and/or gesture module 8 with processors40, communication unit 44, storage device 48, input device 42, andoutput device 46. UI module 6 and gesture module 8 may each includeprogram instructions and/or data that are executable by computing device2. As one example, UI module 6 may include instructions that causecomputing device 2 to perform one or more of the operations and actionsdescribed in the present disclosure.

Computing device 2 may include active beam 54. Active beam 54, in someexamples, is configured to store one or more tokens (e.g., one or moreword-level tokens and/or phrase-level tokens) generated by gesturemodule 8. Active beam 54 may be included within storage devices 48. Thespecific functionality of active beam 54 is further described in thedescription of FIG. 3, below.

Computing device 2 may also include next beam 56. Next beam 56, in someexamples, is configured to store one or more tokens generated by gesturemodule 8 (e.g., one or more word-level tokens and/or phrase-leveltokens). Next beam 56 may be included within storage devices 48. Thespecific functionality of next beam 56 is further described in thedescription of FIG. 3, below.

Computing device 2 can include additional components that, for clarity,are not shown in FIG. 2. For example, computing device 2 can include abattery to provide power to the components of computing device 2.Similarly, the components of computing device 2 shown in FIG. 2 may notbe necessary in every example of computing device 2. For example, insome configurations, computing device 2 may not include communicationunit 44.

In accordance with the techniques of this disclosure, computing device 2may output a graphical keyboard comprising a plurality of keys at outputdevice 46. User 18 may perform a gesture to select a group of keys ofthe plurality of keys at input device 42. In response to user 18performing the gesture, input device 42 may detect a gesture path, suchas gesture path 22 of FIG. 1, which may be received by UI module 6 asgesture path data. The gesture path may include a first portion of thegesture to select a first key of the plurality of keys (e.g., portion22A of gesture path 22) and a second portion of the gesture to select asecond key of the plurality of keys (e.g., portion 22B of gesture path22). Gesture module 8 may receive the gesture path data from UI module6. In some examples, UI module 6 incrementally sends the gesture pathdata to gesture module 8 as gesture path 22 is detected by input device42.

In response to receiving the gesture path data, gesture module 8 maycreate a token at the entry node of a lexicon which may be included inlanguage model 10. In some examples, language module 10 may beimplemented as a trie data structure. Each movable token may represent apartial alignment between a node in the lexicon (i.e., a partial wordand/or phrase) and a point along the gesture. As the token advances tochild nodes in the lexicon (i.e., next letters in the word and/or nextwords of a phrase) the corresponding alignment point on the gesture mayadvance as well. As the token advances to the next letter in a word orto the next word in a phrase, techniques of the disclosure may determinehow far the token needs to advance along the gesture path. For instance,techniques of the disclosure may include searching for an alignmentpoint along the gesture that best aligns to a letter of a key, takinginto account a number of features described below.

As described in FIG. 1, a lexicon trie data structure may contain aplurality of nodes, each node may represent a letter. Gesture module 8may push the token into active beam 54. Gesture module 8 may create atoken copy on each of the token's child nodes. In the example of FIG. 1,gesture module 8 may create a first word-level token copy on the childnode representing the letter “W” (e.g., corresponding to a predicted keyselection of “W” key 20B) and a second token copy on the child noderepresenting the letter “Q” (e.g., corresponding to a predicted keyselection of “Q” key 20A). Each of the word-level tokens may include asingle string of predicted characters.

For each token copy, gesture module 8 may determine, based on aplurality of features associated with the gesture path data, analignment point traversed by the gesture. In the example of FIG. 1,gesture module 8 may determine that a first alignment point is locatedat the start of gesture path 22. In some examples, gesture module 8 maydetermine the curvature of the path at a point along the gesture path.In such examples, gesture module 8 may determine that the point is morelikely to be an alignment point where there is a high curvature (wherethe gesture path changes direction abruptly at the point). In otherexamples, gesture module 8 may determine a mid-segment curvature (themaximum curvature of the gesture path between two points along thegesture). In another example, gesture module 8 may determine that apoint is less likely to be the next alignment point where there is ahigh mid-segment curvature. In some examples, gesture module 8 maydetermine that a point is an alignment point based on the speed at whichthe gesture path was detected. In some examples, a slower rate ofdetection indicates that the point is an alignment point. In someexamples, a high mid-segment curvature may indicate that there werecorners between a first point and a second point, suggesting that thesecond point is less likely to be the next alignment point (i.e., apoint was missed in-between).

In some examples, an alignment point may be based on the maximumdistance between points of a gesture segment between two or more pointsand an ideal line from a first key to a second key. An ideal line maybe, e.g., a shortest distance path from the first key to the second key.For a better alignment the maximum distance may be small, signifyingthat the gesture segment does not deviate from the ideal line.

For each alignment point, gesture module 8 may determine respective costvalues for each of at least two keys of the plurality of keys. Each ofthe respective cost values may represent a probability that thealignment point indicates a key of the plurality of keys. In the exampleof FIG. 1, gesture module 8 may determine a first cost valuerepresenting a probability that the first alignment point indicates thenode representing the letter “W” and a second cost value representing aprobability that the first alignment point indicates the noderepresenting the letter “Q”. In some examples, gesture module 8 may thenupdate the token copy with the respective alignment point and/or costvalue and push the token copy in next beam 56. In the example of FIG. 1,gesture module 8 may add the first cost value to the first token copyand the second cost value to the second token copy.

In some examples, gesture module 8 determines the respective cost valuesby comparing respective physical cost values with respective lexicalcost values, as further described below. In some examples, gesturemodule 8 may apply one or more weighting factors to the respectivephysical cost values, and may apply one or more different weightingfactors to the respective lexical cost values. For instance, gesturemodule 8 may determine a cost value by summing the result of multiplyinga physical cost value by a physical weighting factor, and multiplying alexical cost value by a lexical weighting factor.

In some examples, gesture module 8 may determine that one or morelexical weighting factors applied to the one or more lexical cost valuesshould be greater in magnitude than a magnitude of one or morerespective physical weighting factors applied to the one or morephysical cost values, such as where the gesture path is detected at highrate of speed. For instance, gesture module 8 may determine that a valueassociated with a feature (e.g., speed) satisfies one or morethresholds, such as when a global speed of the gesture is greater thanor equal to a threshold value, less than or equal to a threshold value,etc. In certain examples, gesture module 8 may determine that thephysical cost values are unreliable if the determined value satisfies athreshold. In some examples, gesture module 8 may use statisticalmachine learning to adapt to the style of the user and modify theweighting values over time. For instance, gesture module 8 may, inresponse to determining that the user is inaccurate while performinggestures, weight the lexical cost values greater than the physical costvalues. In some examples, gesture module 8 may determine that thephysical cost values should be weighted greater than the lexical costvalues. Gesture module 8 may determine that the physical cost valuesshould be weighted greater than the lexical cost values where there isan indication that the lexical cost values may be unreliable, such aswhere the user has a history of entering words not included in thelexicon. In some examples, the weighting values may be estimated andoptimized heuristically, such as by measuring accuracy from a pluralityof computing devices.

Gesture module 8 may determine respective physical cost values for eachof the at least two keys of the plurality of keys. Each of therespective physical cost values may represent a probability thatphysical features of an alignment point of the group of alignment pointsindicate physical features of a key of the plurality of keys. Forinstance, gesture module 8 may determine the respective physical costvalues by evaluating the Euclidian distance between an alignment pointof the group of alignment points and a keyboard position of key.

Physical features of the plurality of keys may be included in keyregions 52. For example, key regions 52 may include, for each of theplurality of keys, a set of coordinates that correspond to a locationand/or area of graphical keyboard 16 where each key is displayed. In theexample of FIG. 1, gesture module 8 may determine a first physical costvalue based on the Euclidian distance between the first alignment pointand “W” key 20B. In some examples, gesture module 8 may determine thephysical cost values by comparing the Euclidian distance between a firstalignment point and a second alignment point with the Euclidian distancebetween a first letter indicated by the first alignment point and asecond letter which may be represented by the second alignment point.Gesture module 8 may determine that the cost value of the second letteris inversely proportional to the difference between the distances (i.e.,that the second letter is more probable where the distances are moresimilar). For instance, a smaller distance may suggest a betteralignment.

Gesture module 8 may also determine the respective cost values bydetermining respective lexical cost values for each of the at least twokeys of the plurality of keys. Each of the respective lexical costvalues may represent a probability that a letter represented by a key ofthe plurality of keys is included in a candidate word based on theword-level token. The lexical cost values may be based on language model10. For instance, the lexical cost values may represent the likelihoodthat a given letter is selected based on probable words included inlanguage model 10. In the example of FIG. 1, gesture module 8 maydetermine a first lexical cost value based on an entry in language model10 indicating a frequency that the letter “W” is the first letter in aword.

Gesture module 8 may determine whether the token is at a terminal nodeof the lexicon. A terminal node of the lexicon may be a node thatrepresents a complete word included in the lexicon. For instance, in theexample of FIG. 1, gesture module 8 may determine, based at least inpart on the indication of portion 22A of gesture path 22, a word-leveltoken including the single string of predicted characters “we”. Such aword-level token may represent a prefix of one or more candidate wordsincluded in the lexicon, such as the words “went”, “were”, “weather”,and the like. In addition, the word-level token may itself represent acomplete word included in the lexicon, such as the word “we” in theEnglish language. In response to determining that the word-level tokenrepresents a candidate word included in the lexicon, gesture module 8may generate a next-word token. The next-word token may indicate that anext selected key of the plurality of keys is a prefix of a secondword-level token.

As such, in response to receiving an indication of a portion of thegesture to select a next key of the plurality of keys, gesture module 8may create a token copy on each of the word-level token's child nodes toinclude a predicted character indicated by the received portion of thegesture. In addition, in response to receiving the indication of theportion of the gesture to select the next key of the plurality of keys,gesture module 8 may create a phrase-level token that includes a secondword-level token including a second string of predicted characters forwhich the predicted character is a prefix. Gesture module 8 maydetermine the phrase-level token as a combination of the firstword-level token that represents the candidate word included in thelexicon and the second word-level token. Accordingly, gesture module 8may incrementally determine one or more candidate words indicated by thegesture and one or more candidate phrases indicated by the gesture inparallel.

As an example, gesture module 8 may receive an indication of portion 22Aof gesture path 22. In response, gesture module 8 may determine a firstword-level token including the single string of predicted characters“we”. Gesture module 8 may determine one or more candidate wordsindicated by the gesture using the first word-level token. For instance,gesture module 8 may determine one or more candidate words for which thesingle string of predicted characters is a prefix, such as the words“were”, “went”, and the like. In addition, gesture module 8 maydetermine that the first word-level token including the single string ofpredicted characters “we” represents a candidate word included in thelexicon, such as the word “we” in the English language. In response,gesture module 8 may generate a next-word token that indicates that anext selected key is a prefix of a second word-level token.

In the present example, as the user continues to perform the gesture,gesture module 8 may receive an indication of portion 22B of gesturepath 22. In response, gesture module 8 may create a token copy on eachof the word-level token's child nodes to include a predicted characterindicated by the received portion of the gesture, such as the letter “a”corresponding to a predicted selection of “A” key 20E. As such, gesturemodule 8 may advance the first word-level token to include the singlestring of predicted characters “wea”. Based on the first word-leveltoken, gesture module 8 may determine one or more candidate wordsindicated by the gesture, such as words included in the lexicon forwhich the single string of predicted characters “wea” is a prefix (e.g.,the words “wear”, “weather”, and the like).

In addition, in the present example, gesture module 8 may determine, inresponse to generating the next-word token that indicates that a nextselected key is a prefix of a second word-level token, a secondword-level token that includes the predicted character “a” correspondingto the predicted selection of “A” key 20E. In this example, gesturemodule 8 may determine a phrase-level token as a combination of thefirst word-level token including the single string of predictedcharacters “we” and the second word-level token including the singlestring of predicted characters “a”. Gesture module 8 may determine oneor more candidate words for which the single string of predictedcharacters “a” included in the second word-level token is a prefix, suchas the words “are”, “am”, and the like. Gesture module 8 may determineone or more candidate phrases indicated by the gesture as a combinationof the first word-level token that represents the candidate word in thelexicon (i.e., the word “we” in the present example) and the one or morecandidate words for which the single string of predicted charactersincluded in the second word-level token is a prefix (e.g., the words“are”, “am”, etc.) Gesture module 8 may, in certain examples, determinethe one or more candidate phrases indicated by the gesture using thelexical model (e.g., language model 10), such as by determining aprobability that a given candidate word associated with the secondword-level token follows the candidate word associated with the firstword-level token. For instance, in this example, gesture module 8 maydetermine that a candidate word “are” is more likely to follow thecandidate word “we” than the candidate word “am”, as the phrase “we are”may be more have a higher probability in language model 10 than thephrase “we am”.

In certain examples, gesture module 8 may maintain a threshold number ofword-level and/or phrase-level tokens (e.g., fifty tokens, one hundredtokens, two hundred tokens, or other numbers of tokens) and discard therest. For instance, gesture module 8 may maintain a group of the onehundred word-level and/or phrase-level tokens that include the mostlikely words and/or character strings indicated by the gesture, asdetermined based on the spatial and language models. In this way,gesture module 8 may efficiently scale to large lexicons.

In some examples, gesture module 8 may select a character stringincluded in the phrase-level token as a committed character string, andmay output the committed character string for display at thepresence-sensitive display, such as at committed-text region 14 of GUI12. For example, a user may provide a gesture at graphical keyboard 16to select a group of keys corresponding to the phrase “this is adelightful keyboard”. In response to receiving the indication of thegesture, gesture module 8 may determine one or more phrase level tokensincluding a plurality of predicted character strings. Gesture module 8may determine that a phrase-level token satisfies a threshold, such as athreshold number of word-level tokens included in the phrase-leveltoken. In response to determining that the phrase-level token satisfiesthe threshold, gesture module 8 may select one or more character stringsincluded in the phrase-level token as committed character strings. Forinstance, in the present example, gesture module 8 may select thecharacter strings “this is a” as committed character strings. Inresponse, gesture module 8 may remove the one or more committedcharacter strings from the plurality of character strings included inthe phrase-level token, and may output the one or more committedcharacter strings for display at committed-text region 14 of GUI 12. Inaddition, gesture module 8 may output the phrase-level token from whichthe committed character strings have been removed for display at one ormore of text-suggestion regions 24.

For instance, in this example, prior to determining that a phrase-leveltoken including the plurality of predicted character strings “this is adelightful keyboard” satisfies the threshold, gesture module 8 mayoutput the predicted phrase “this is a delightful keyboard” for displayat one or more of text-suggestion regions 24. In response to determiningthat the phrase-level token satisfies the threshold (e.g., determiningthat a number of word-level tokens included in the phrase-level tokensatisfies a threshold number, such as five word-level tokens), gesturemodule 8 may select one or more character strings as committed characterstrings, such as the character strings “this is a”. In this example,gesture module 8 may remove the character strings “this is a” from thephrase-level token, thereby determining a modified phrase-level tokenthat includes the character strings “delightful keyboard”. Gesturemodule 8 may output the committed character strings “this is a” fordisplay at committed-text region 14 of GUI 12. In addition, gesturemodule 8 may output the modified phrase-level token for display at oneor more of text-suggestion regions 24, such that the character strings“this is a delightful keyboard” is no longer displayed attext-suggestion regions 24 and the character strings “delightfulkeyboard” is displayed at one or more of text-suggestion regions 24.

Gesture module 8 may determine whether UI module 6 has completedreceiving the gesture path data. Where UI module 6 has completedreceiving the gesture path data, gesture module 8 may output one or morecandidate words and/or phrases for display at the presence-sensitivedisplay. Where UI module 6 has not completed receiving the gesture pathdata, gesture module 8 may continue to incrementally process the gesturepath data. In some examples, gesture module 8 may output one or moreoutput predictions prior to UI module 6 completing reception of thegesture path data.

As such, according to techniques of this disclosure, gesture module 8may determine one or more word-level tokens and one or more phrase-leveltokens based on a received indication of a gesture to select one or morekeys of a graphical keyboard, thereby enabling a user to enter a word orphrase by providing one continuous motion gesture. In addition, byincrementally determining multiple word-level and/or phrase level tokensand advancing the respective tokens as gesture module 8 receives theindication of the gesture to select the group of keys, gesture module 8may incrementally update its determination of candidate words and/orphrases based on spatial and language models, thereby enabling a moreaccurate interpretation of the gesture. Moreover, by enabling the userto provide one continuous gesture to enter multi-word phrases,techniques of this disclosure may more effectively recognize ambiguoushyphenated or connected words. For instance, hyphenated or connectedwords such as “smoke-free” may be included in a lexicon as the singlewords “smoke” and “free”, or as the hyphenated word “smoke-free”. Byenabling a user to provide a single continuous gesture to entermulti-word phrases, techniques described herein may enable the user toprovide the single gesture to enter the words without regard to whetherthe word is hyphenated.

FIGS. 3A-C are block diagrams illustrating further details of oneexample of a computing device shown in FIG. 1, in accordance with one ormore techniques of the present disclosure. As shown in the conceptualexample of FIG. 3A, at time 60, computing device 2 may include GUI 12,active beam 54A, and next beam 56A. GUI 12 may include graphicalkeyboard 16 which may include “Q” key 20A and “W” key 20B. While shownin FIG. 3A, gesture path 22A and/or alignment point 21A may not bevisible during the performance of the techniques described herein.

As shown in the example of FIG. 1, a user may desire to enter text intocomputing device 2 by performing a gesture at graphical keyboard 16. Aspreviously discussed, while the user performs the gesture, computingdevice 2 may detect a gesture having a gesture path. In the example ofFIG. 3A, computing device 2 is shown as having detected gesture path22A.

In response to detecting gesture path 22A, computing device 2 maydetermine alignment point 21A along gesture path 22A. Additionally, inresponse to detecting gesture path 22A, computing device 2 may create aword-level token and push the word-level token into active beam 54A. Attime 60, the contents on active beam 54A may be represented by Table 1below.

TABLE 1 Parent Letter Key of Letter Index Index Current Node Chain CostValue 0 — — — 0

In Table 1, each row represents an individual word-level token, theindex column represents a unique identifier for each word-level token,the parent index column represents the index value of the word-leveltoken to which the listed word-level token is a child, the letter key ofthe current node column represent the letter key represented by thecurrent node of the word-level token, the letter chain column representsall of the letter keys represented by the nodes from the entry node tothe current node of the word-level token, and the cost value columnrepresent the cost value of the word-level token. As shown in Table 1,the created word-level token has an index of 0 (i.e., word-leveltokeno), no parent index, no letter key of the current node, no letterchain, and a cost value of zero.

To determine the text indicated by the gesture, computing device 2 maycreate a copy of each word-level token on its child nodes. In someexamples, an entry node may have 26 child nodes (one for each letter ofthe English alphabet). For simplicity, in the example of FIG. 3A, theentry node has only two child nodes on the letters “W” and “Q”.Therefore, computing device 2 may create a copy of the word-level tokenwith index 0 on child node “W” (i.e., word-level token′) and child node“Q” (i.e., word-level token). For each created word-level token copy,computing device 2 may determine a cost value as described above.Computing device 2 may push each word-level token copy to next beam 56A,the contents of which may be represented by Table 2 below.

TABLE 2 Parent Letter Key of Letter Index Index Current Node Chain CostValue 1 0 W W CV1 2 0 Q Q CV2

The entries shown in Table 2 are similar in format to the entry shown inTable 1. In Table 2, word-level token′ has cost value CV1 and word-leveltoken.sub.2 has cost value CV2. After creating the word-level tokencopies, computing device 2 may determine that word-level tokeno is not aterminal node, and may discard word-level tokeno. Computing device 2 maysubsequently determine whether active beam 54A is empty (i.e., containsno tokens). In response to determining that active beam 54A is empty,computing device 2 may copy the contents of next beam 56A to active beam54B of FIG. 3B and discard the contents of next beam 56A.

In the example of FIG. 3B, computing device 2 is shown as havingdetected gesture path 22B at time 62. As described above, the contentsof active beam 54B may be represented by Table 2. Computing device 2 maydetermine alignment point 21B along gesture path 22B. Computing device 2may, for each word-level token in active beam 54B, create a copy on eachchild node. In the example of FIG. 3B, word-level token′ and word-leveltoken.sub.2 each have child nodes with letter keys “E” and “R”. For eachcreated word-level token copy, computing device 2 may determine a costvalue as described above. Computing device 2 may push each word-leveltoken copy into next beam 56B, the contents of which may be representedby Table 3 below.

TABLE 3 Parent Letter Key of Letter Index Index Current Node Chain CostValue 3 1 E WE CV1 + CV3 4 1 R WR CV1 + CV4 5 2 E QE CV2 + CV5 6 2 R QRCV2 + CV6

The entries shown in Table 3 are similar in format to the entries shownin Table 1 and Table 2. In Table 3, the cost value for each word-leveltoken includes the cost value for the previous letters and the costvalue for the current letter. Computing device 2 may determine which, ifany, of the word-level tokens are on terminal nodes. For instance,computing device 2 may determine that word-level token.sub.3 is on aterminal node because its string of predicted characters (i.e., itsletter chain) “WE” represents a candidate word included in the lexicon(e.g., the English language).

In response to determining that a word-level token is on a terminalnode, computing device 2 may copy the word-level token to a list ofoutput predictions. The list of output predictions may be represented byTable 4 below. In some examples, computing device 2 may copy only theletter chain of the token to the list of output predictions.

TABLE 4 Parent Letter Key of Letter Index Index Current Node Chain CostValue 3 1 O NO CV1 + CV3

In addition, computing device 2 may generate, in response to determiningthat the word-level token is on a terminal node, a next-word token thatindicates that a next selected key of the plurality of keys is a prefixof a second word-level token. The next-word token may be considered anentry node of the second word-level token. Computing device 2 may pushthe next-word token (i.e., the entry node of the second word-leveltoken) into active beam 54B, the contents of which may be represented byTable 5 below.

TABLE 5 Parent Letter Key of Letter Index Index Current Node Chain CostValue 3 1 E WE CV1 + CV3 4 1 R WR CV1 + CV4 5 2 E QE CV2 + CV5 6 2 R QRCV2 + CV6 7 3 — — 0

The entries shown in Table 5 are similar in format to the entries shownin Tables 1, 2, and 3. As shown in Table 5, the created word-level tokencorresponding to the next-word token has an index of 7 (i.e., word-leveltoken.sub.7), a parent index of 3 (i.e., corresponding to word-leveltoken.sub.3), no letter key of the current node, no letter chain, and acost value of zero. In certain examples, the combination of word-leveltoken.sub.3 and word-level token.sub.7 may be considered a phrase-leveltoken that includes a plurality of character strings.

In the example of FIG. 3C, computing device 2 is shown as havingdetected gesture path 22B at time 74. As described above, the contentsof active beam 54C may be represented by Table 5. Computing device 2 maydetermine alignment point 21C along gesture path 22B. Computing device 2may, for each word-level token in active beam 54C, create a copy on eachchild node. In the example of FIG. 3C, token.sub.3 through token.sub.6each have child nodes with letter keys “E” and “R”. In addition,token.sub.7 may be considered an entry node for a second word-leveltoken of a phrase-level token including token.sub.3. As such, computingdevice 2 may create a copy at a child node of word-level token.sub.7included in the phrase-level token. For each created word-level tokencopy and each phrase-level token, computing device 2 may determine acost value as described above. Computing device 2 may push eachword-level token copy and each phrase-level token copy into next beam56C, the contents of which may be represented by Table 6 below.

TABLE 6 Parent Letter Key of Letter Index Index Current Node Chain CostValue 8 3 A WEA CV1 + CV3 + CV7 9 3 S WES CV1 + CV3 + CV8 10 4 A WRACV1 + CV4 + CV9 11 4 S WRS CV1 + CV4 + CV10 12 5 A QEA CV2 + CV5 + CV1113 5 S QES CV2 + CV5 + CV12 14 6 A QRA CV2 + CV6 + CV13 15 6 S QRS CV2 +CV6 + CV14 16 7 A A CV15 17 7 S S CV16 18 16 A WE A CV1 + CV3 + CV15 1917 S WE S CV1 + CV3 + CV16

The entries shown in Table 6 are similar in format to the entries shownin Tables 1-3. In Table 6, the cost value for each word-level tokenincludes the cost value for the previous letters and the cost value forthe current letter. In addition, the cost value for each phrase-leveltoken (i.e., phrase-level token's and phrase-level token.sub.19)includes the cost value for the previous letters of the each previousword-level token, the cost value for each previous letter in the currentword-level token, and the cost value for the current letter of thecurrent word-level token. As such, computing device 2 may determine oneor more phrase-level tokens based at least in part on a word-level tokenthat represents a candidate word included in the lexicon and a predictedselection of a character key. In such a way, computing device 2 maydetermine both word-level tokens and phrase-level tokens in response toreceiving an indication of a gesture to select a group of keys includedin a graphical keyboard. Computing device 2 may continue toincrementally determine the one or more word-level tokens and one ormore phrase-level tokens as computing device 2 receives furtherindications of the gesture, thereby enabling a user to provide a singlegesture to select a group of keys of a word or phrase.

In the example of FIG. 3C, computing device 2 may determine whether theuser has completed performing the gesture. In response to determiningthat the user has completed performing the gesture, computing device 2may output a list of output predictions. The output predictions mayinclude one or more candidate words based at least in part on the one ormore word-level tokens, one or more candidate phrases based at least inpart on the one or more phrase-level tokens, or both. In some examples,computing device 2 may determine a subset of the list of outputpredictions which have the highest cost values (i.e., the predictionswith the highest probability). Additionally, in some examples, computingdevice 2 may, at each subsequent alignment point, revise the cost valuesof the tokens contained in the list of output predictions. For instance,computing device 2 may increase the cost value of token.sub.3 (e.g.,make token.sub.3 less probable) in response to detecting gesture path22B.

FIGS. 4A-B are flow diagrams illustrating example operations of acomputing device to determine a candidate word and/or phrase from agesture, in accordance with one or more techniques of the presentdisclosure. For purposes of illustration only, the example operationsare described below within the context of computing device 2, as shownin FIGS. 1 and 2.

In the example of FIGS. 4A-B, computing device 2 may initially output agraphical keyboard (e.g., graphical keyboard 16) comprising a pluralityof keys at a presence-sensitive display (e.g., UI device 4) of computingdevice 2 (70). Computing device 2 may detect a gesture at thepresence-sensitive display (72). For example, UI device 4 may detect agesture having gesture path 22. UI module 6, executing on one or moreprocessors 40, may receive an indication of the gesture detected at UIdevice 4. In response to receiving the indication of the gesturedetected at the presence-sensitive display, computing device 2 maygenerate a word-level token having a cost value of zero at the entrynode of a lexicon stored on computing device 2 as a lexicon trie (74).Computing device 2 may push the word-level token into an active beam(76). For instance, gesture module 8, executing on one or moreprocessors 40, may push the word-level token into active beam 54.

Computing device 2 may select a word-level token from the active beam(78). Computing device 2 may create a copy of the word-level token oneach child node of the word-level token (80). Computing device 2 mayselect a word-level token copy (82) and determine an alignment pointalong the gesture (84). Computing device 2 may determine a cost valuerepresenting a probability that the alignment point indicates the letterkey of the node on which the word-level token copy is positioned and addthe cost value to the word-level token copy (86). Computing device 2 maypush the word-level token copy into a next beam (88). Computing device 2may determine whether there are any word-level token copies remaining(90). If there are word-level token copies remaining (94), computingdevice 2 may select a new word-level token copy (82).

If there are not any word-level token copies remaining (92), computingdevice 2 may determine whether the word-level token is at a terminalnode of the lexicon trie (96). For instance, gesture module 8 may uselanguage model 10 to determine whether the word-level token represents acandidate word included in a lexicon (e.g., the English language). Ifthe word-level token is at a terminal node (98), computing device 2 maycopy the word represented by the word-level token to a list of candidatewords (102), and may generate a next-word token that indicates that anext selected key of the plurality of keys of the graphical keyboard isa prefix of a second word-level token (103). After copying the word tothe list of candidate words and generating the next-word token, or ifthe word-level token is not at a terminal node (100), computing device 2may discard the word-level token (104).

Computing device 2 may determine whether any word-level tokens remain inthe active beam (106). If there are word-level tokens remaining in theactive beam (110), computing device 2 may select a new word-level tokenfrom the active beam (78). If there are no word-level tokens remainingin the active beam (108), computing device 2 may determine whether anyword-level tokens remain in the next beam (112). If there are word-leveltokens remaining in the next beam (114), computing device 2 may copy thenext beam to the active beam (120) and select a new word-level tokenfrom the active beam (78). If there are no word-level tokens remainingin the next beam (116), computing device 2 may output the list ofcandidate words at the presence-sensitive display (118).

FIG. 5 is a flow diagram illustrating example operations of a computingdevice to determine a candidate word and/or phrase from a gesture, inaccordance with one or more techniques of the present disclosure. Forpurposes of illustration only, the example operations are describedbelow within the context of computing device 2, as shown in FIGS. 1 and2.

In the example of FIG. 5, computing device 2 may output, for display ata presence-sensitive display operatively coupled to the computingdevice, a graphical keyboard comprising a plurality of keys (130). Forexample, UI module 6, executing on one or more processors 40, may outputgraphical keyboard 16 for display at output device 46. Computing device2 may receive an indication of a gesture detected at thepresence-sensitive display, a first portion of the gesture to select afirst key of the plurality of keys and a second portion of the gestureto select a second key of the plurality of keys (132). For instance, UImodule 6 may receive an indication of a gesture detected at input device42, the gesture including portion 22A of gesture path 22 to select “E”key 20C and portion 22B of gesture path 22 to select “A” key 20E.

Computing device 2 may determine, based at least in part on the firstkey, a word-level token comprising a single string of a plurality ofpredicted characters (134). As an example, gesture module 8, executingon one or more processors 40, may determine a word-level token includingthe single string “we” based at least in part on “E” key 20C. Computingdevice 2 may determine that the word-level token represents a candidateword included in a lexicon (136). For instance, gesture module 8 maydetermine that the word-level token including the single string “we”represents the candidate word “we” included in a lexicon (e.g., theEnglish language) of language model 10.

Computing device 2 may determine, in response to determining that theword-level token represents the candidate word in the lexicon, aphrase-level token based at least in part on the word-level token andthe second key (138). The phrase-level token may include a plurality ofcharacter strings. For example, gesture module 8 may determine aphrase-level token as the plurality of character strings “we” and “a”,based at least in part on the word-level token “we” and “A” key 20E.

In one example, the operations include determining, by computing device2 and in response to determining the phrase-level token, a candidatephrase based at least in part on the phrase-level token; and outputting,by computing device 2, the candidate phrase for display at thepresence-sensitive display. In one example, the plurality of characterstrings of the phrase-level token comprises a first character string anda second character string, wherein the word-level token comprises thefirst character string, and wherein the second character stringcomprises a character associated with the second key. In one example,the second character string begins with the character associated withthe second key.

In one example, the word-level token comprises a first word-level tokencomprising a first single string of the plurality of predictedcharacters, the operations further comprising determining, by thecomputing device and based at least in part on the first key and thesecond key, a second word-level token comprising a second single stringof the plurality of predicted characters. In one example, the word-leveltoken is a first word-level token comprising a first single string ofthe plurality of predicted characters, and determining the phrase-leveltoken comprises: generating, in response to determining that the firstword-level token represents the candidate word in the lexicon, anext-word token that indicates that a next selected key of the pluralityof keys is a prefix of a second word-level token; determining, inresponse to determining the next-word token, a second word-level tokencomprising a second single string of the plurality of predictedcharacters, wherein the second key is a prefix of the second word-leveltoken; and determining the phrase-level token as a combination of thefirst word-level token and the second word-level token.

In one example, determining the phrase-level token further includes:determining, based on a plurality of features associated with thegesture, a group of alignment points traversed by the gesture;determining respective cost values for each of at least the first key,the second key, and a third key, wherein each of the respective costvalues represents a probability that an alignment point of the group ofalignment points indicates a key of the plurality of keys; determining afirst combined cost value based at least in part on the determined costvalue for the first key and the determined cost value for the secondkey; determining a second combined cost value based at least in part onthe determined cost value for the first key and the determined costvalue for the third key; comparing the first combined cost value and thesecond combined cost value; and determining the second word-level tokenbased on the comparison of the first combined cost value and the secondcombined cost value.

In one example, the plurality of features associated with the gesturecomprises at least one of: a length of a segment of the gesture, whereinthe segment comprises a path traversed by the gesture at thepresence-sensitive display; a direction of the segment of the gesture; acurvature of the segment of the gesture; a local speed that represents arate at which the segment of the gesture was detected; and a globalspeed that represents a rate at which the gesture was detected. In oneexample, determining the respective cost values for each of at least thefirst key, the second key, and the third key comprises: determiningrespective physical cost values for each of at least the first key, thesecond key, and the third key, wherein each of the respective physicalcost values represents a probability that at least one physical featureof an alignment point of the group of alignment points indicates atleast one physical feature of a key of the plurality of keys;determining respective lexical cost values for each of at least thefirst key, the second key, and the third key, wherein each of therespective lexical cost values represents a probability that a letterrepresented by a key of the plurality of keys is included in a candidateword; and determining the respective cost values for each of at leastthe first key, the second key, and the third key based on the respectivephysical cost values and the respective lexical cost values for each ofat least the first key, the second key, and the third key.

In one example, determining the respective physical cost values for eachof at least the first key, the second key, and the third key comprisescomparing key regions of each of at least the first key, the second key,and the third key with at least one of the plurality of featuresassociated with the gesture, wherein the key regions comprise locationsof the presence-sensitive display that output the respective keys. Inone example, determining the respective lexical cost values for each ofat least the first key, the second key, and the third key comprisescomparing each of at least the first key, the second key, and the thirdkey with a language model. In one example, the language model comprisesan n-gram language model. In one example, language model comprises agroup of predefined phrases.

In one example, the phrase-level token is a first phrase-level tokencomprising a first plurality of character strings, and the operationsfurther include: determining, by computing device 2, that the firstphrase-level token satisfies a threshold; responsive to determining thatthe first phrase-level token satisfies the threshold, selecting, bycomputing device 2, a character string of the first plurality ofcharacter strings of the first phrase-level token as a committedcharacter string; and removing, by computing device 2, the committedcharacter string from the first plurality of character strings of thephrase-level token to determine a second phrase-level token comprising asecond plurality of character strings.

In one example, the operations further include: outputting, by computingdevice 2, the first phrase-level token for display at a text-suggestionregion of the presence-sensitive display; and responsive to determiningthat the first phrase-level token satisfies the threshold: outputting,by computing device 2, the committed character string for display at acommitted-text region of the presence-sensitive display, wherein thecommitted-text region is different than the text-suggestion region; andoutputting, by computing device 2, the second plurality of characterstrings for display at the text-suggestion region. In one example,determining that the first phrase-level token satisfies the thresholdcomprises determining that a number of character strings of the firstplurality of character strings satisfies a threshold number of characterstrings. In one example, receiving the indication of the gesturecomprises receiving an indication of a motion of an input unit from afirst location of the presence-sensitive display to a second location ofthe presence-sensitive display with substantially constant contactbetween the input unit and the presence-sensitive display. In oneexample, the lexicon is implemented, by computing device 2, as a triedata structure.

FIG. 6 is a block diagram illustrating an example computing device thatoutputs graphical content for display at a remote device, in accordancewith one or more techniques of the present disclosure. Graphicalcontent, generally, may include any visual information that may beoutput for display, such as text, images, a group of moving images, etc.The example shown in FIG. 6 includes a computing device 140,presence-sensitive display 144, communication unit 150, projector 160,projector screen 162, tablet device 166, and visual display device 170.Although shown for purposes of example in FIGS. 1 and 2 as a stand-alonecomputing device, a computing device may, generally, be any component orsystem that includes a processor or other suitable computing environmentfor executing software instructions and, for example, need not include apresence-sensitive display.

As shown in the example of FIG. 6, computing device 140 may be aprocessor that includes functionality as described with respect toprocessor 40 in FIG. 2. In such examples, computing device 140 may beoperatively coupled to presence-sensitive display 144 by a communicationchannel 142A, which may be a system bus or other suitable connection.Computing device 140 may also be operatively coupled to communicationunit 150, further described below, by a communication channel 142B,which may also be a system bus or other suitable connection. Althoughshown separately as an example in FIG. 6, computing device 140 may beoperatively coupled to presence-sensitive display 144 and communicationunit 150 by any number of one or more communication channels.

In other examples, such as illustrated previously in FIGS. 1-2,computing device 140 may be a portable or mobile device such as mobilephones (including smart phones), laptop computers, etc. In someexamples, computing device 140 may be a desktop computers, tabletcomputers, smart television platforms, cameras, personal digitalassistants (PDAs), servers, mainframes, etc.

Presence-sensitive display 144, as shown in FIG. 6, may include displaydevice 146 and presence-sensitive input device 148. Display device 146may, for example, receive data from computing device 140 and display thegraphical content. In some examples, presence-sensitive input device 148may determine one or more user inputs (e.g., continuous gestures,multi-touch gestures, single-touch gestures, etc.) at presence-sensitivedisplay 144 using capacitive, inductive, and/or optical recognitiontechniques and send indications of such user input to computing device140 using communication channel 142A. In some examples,presence-sensitive input device 148 may be physically positioned on topof display device 146 such that, when a user positions an input unitover a graphical element displayed by display device 146, the locationat which presence-sensitive input device 148 corresponds to the locationof display device 146 at which the graphical element is displayed.

As shown in FIG. 6, computing device 140 may also include and/or beoperatively coupled with communication unit 150. Communication unit 150may include functionality of communication unit 44 as described in FIG.2. Examples of communication unit 150 may include a network interfacecard, an Ethernet card, an optical transceiver, a radio frequencytransceiver, or any other type of device that can send and receiveinformation. Other examples of such communication units may includeBluetooth, 3G and WiFi radios, Universal Serial Bus (USB) interfaces,etc. Computing device 140 may also include and/or be operatively coupledwith one or more other devices, e.g., input devices, output devices,memory, storage devices, etc. that are not shown in FIG. 6 for purposesof brevity and illustration.

FIG. 6 also illustrates a projector 160 and projector screen 162. Othersuch examples of projection devices may include electronic whiteboards,holographic display devices, and any other suitable devices fordisplaying graphical content. Projector 160 and project screen 162 mayinclude one or more communication units that enable the respectivedevices to communicate with computing device 140. In some examples, theone or more communication units may enable communication betweenprojector 160 and projector screen 162. Projector 160 may receive datafrom computing device 140 that includes graphical content. Projector160, in response to receiving the data, may project the graphicalcontent onto projector screen 162. In some examples, projector 160 maydetermine one or more user inputs (e.g., continuous gestures,multi-touch gestures, single-touch gestures, etc.) at projector screenusing optical recognition or other suitable techniques and sendindications of such user input using one or more communication units tocomputing device 140.

Projector screen 162, in some examples, may include a presence-sensitivedisplay 164. Presence-sensitive display 164 may include a subset offunctionality or all of the functionality of UI device 4 as described inthis disclosure. In some examples, presence-sensitive display 164 mayinclude additional functionality. Projector screen 162 (e.g., anelectronic whiteboard), may receive data from computing device 140 anddisplay the graphical content. In some examples, presence-sensitivedisplay 164 may determine one or more user inputs (e.g., continuousgestures, multi-touch gestures, single-touch gestures, etc.) atprojector screen 162 using capacitive, inductive, and/or opticalrecognition techniques and send indications of such user input using oneor more communication units to computing device 140.

FIG. 6 also illustrates tablet device 166 and visual display device 170.Tablet device 166 and visual display device 170 may each includecomputing and connectivity capabilities. Examples of tablet device 166may include e-reader devices, convertible notebook devices, hybrid slatedevices, etc. Examples of visual display device 170 may includetelevisions, computer monitors, etc. As shown in FIG. 6, tablet device166 may include a presence-sensitive display 168. Visual display device170 may include a presence-sensitive display 172. Presence-sensitivedisplays 168, 172 may include a subset of functionality or all of thefunctionality of UI device 4 as described in this disclosure. In someexamples, presence-sensitive displays 168, 172 may include additionalfunctionality. In any case, presence-sensitive display 172, for example,may receive data from computing device 140 and display the graphicalcontent. In some examples, presence-sensitive display 172 may determineone or more user inputs (e.g., continuous gestures, multi-touchgestures, single-touch gestures, etc.) at projector screen usingcapacitive, inductive, and/or optical recognition techniques and sendindications of such user input using one or more communication units tocomputing device 140.

As described above, in some examples, computing device 140 may outputgraphical content for display at presence-sensitive display 144 that iscoupled to computing device 140 by a system bus or other suitablecommunication channel. Computing device 140 may also output graphicalcontent for display at one or more remote devices, such as projector160, projector screen 162, tablet device 166, and visual display device170. For instance, computing device 140 may execute one or moreinstructions to generate and/or modify graphical content in accordancewith techniques of the present disclosure. Computing device 140 mayoutput the data that includes the graphical content to a communicationunit of computing device 140, such as communication unit 150.Communication unit 150 may send the data to one or more of the remotedevices, such as projector 160, projector screen 162, tablet device 166,and/or visual display device 170. In this way, computing device 140 mayoutput the graphical content for display at one or more of the remotedevices. In some examples, one or more of the remote devices may outputthe graphical content at a presence-sensitive display that is includedin and/or operatively coupled to the respective remote devices.

In some examples, computing device 140 may not output graphical contentat presence-sensitive display 144 that is operatively coupled tocomputing device 140. In other examples, computing device 140 may outputgraphical content for display at both a presence-sensitive display 144that is coupled to computing device 140 by communication channel 142A,and at one or more remote devices. In such examples, the graphicalcontent may be displayed substantially contemporaneously at eachrespective device. For instance, some delay may be introduced by thecommunication latency to send the data that includes the graphicalcontent to the remote device. In some examples, graphical contentgenerated by computing device 140 and output for display atpresence-sensitive display 144 may be different than graphical contentdisplay output for display at one or more remote devices.

Computing device 140 may send and receive data using any suitablecommunication techniques. For example, computing device 140 may beoperatively coupled to external network 154 using network link 152A.Each of the remote devices illustrated in FIG. 6 may be operativelycoupled to network external network 154 by one of respective networklinks 152B, 152C, and 152D. External network 154 may include networkhubs, network switches, network routers, etc., that are operativelyinter-coupled thereby providing for the exchange of information betweencomputing device 140 and the remote devices illustrated in FIG. 6. Insome examples, network links 152A-152D may be Ethernet, ATM or othernetwork connections. Such connections may be wireless and/or wiredconnections.

In some examples, computing device 140 may be operatively coupled to oneor more of the remote devices included in FIG. 6 using direct devicecommunication 158. Direct device communication 158 may includecommunications through which computing device 140 sends and receivesdata directly with a remote device, using wired or wirelesscommunication. That is, in some examples of direct device communication158, data sent by computing device 140 may not be forwarded by one ormore additional devices before being received at the remote device, andvice-versa. Examples of direct device communication 158 may includeBluetooth, Near-Field Communication, Universal Serial Bus, WiFi,infrared, etc. One or more of the remote devices illustrated in FIG. 6may be operatively coupled with computing device 140 by communicationlinks 156A-156D. In some examples, communication links 152A-152D may beconnections using Bluetooth, Near-Field Communication, Universal SerialBus, infrared, etc. Such connections may be wireless and/or wiredconnections.

In accordance with techniques of the disclosure, computing device 140may be operatively coupled to visual display device 170 using externalnetwork 154. Computing device 140 may output a graphical keyboard fordisplay at presence-sensitive display 172. For instance, computingdevice 140 may send data that includes a representation of the graphicalkeyboard to communication unit 150. Communication unit 150 may send thedata that includes the representation of the graphical keyboard tovisual display device 170 using external network 154. Visual displaydevice 170, in response to receiving the data using external network154, may cause presence-sensitive display 172 to output the graphicalkeyboard. In response to a user performing a first portion of a gestureat presence-sensitive display 172 to select a first key of the keyboard,visual display device 170 may send an indication of the gesture tocomputing device 140 using external network 154. Communication unit 150may receive the indication of the gesture, and send the indication tocomputing device 140. Similarly, in response to a user performing asecond portion of the gesture at presence-sensitive display 172 toselect a second key of the keyboard, visual display device 170 may sendan indication of the gesture to computing device 140 using externalnetwork 154.

Computing device 140 may determine, based at least in part on the firstkey, a word-level token including a single string of a plurality ofpredicted characters. In some examples, computing device 140 maydetermine that the word-level token represents a candidate work includedin a lexicon. In response to determining that the word-level tokenrepresents the candidate work in the lexicon, computing device 140 maydetermine a phrase-level token based at least in part on the word-leveltoken and the second key. The phrase-level token may include a pluralityof character strings. In certain examples, computing device 140 maydetermine a candidate phrase based at least in part on the phrase-leveltoken. Computing device 140, in some examples, may send data thatincludes the candidate phrase to communication unit 150, which in turnsends the data to visual display device 170 using external network 154.Upon receiving the data, visual display device 170 may causepresence-sensitive display 172 to display the candidate phrase. In thisway, computing device 140 may output the candidate phrase for display atpresence-sensitive screen 172, in accordance with techniques of thedisclosure.

The techniques described in this disclosure may be implemented, at leastin part, in hardware, software, firmware, or any combination thereof.For example, various aspects of the described techniques may beimplemented within one or more processors, including one or moremicroprocessors, digital signal processors (DSPs), application specificintegrated circuits (ASICs), field programmable gate arrays (FPGAs), orany other equivalent integrated or discrete logic circuitry, as well asany combinations of such components. The term “processor” or “processingcircuitry” may generally refer to any of the foregoing logic circuitry,alone or in combination with other logic circuitry, or any otherequivalent circuitry. A control unit including hardware may also performone or more of the techniques of this disclosure.

Such hardware, software, and firmware may be implemented within the samedevice or within separate devices to support the various techniquesdescribed in this disclosure. In addition, any of the described units,modules or components may be implemented together or separately asdiscrete but interoperable logic devices. Depiction of differentfeatures as modules or units is intended to highlight differentfunctional aspects and does not necessarily imply that such modules orunits must be realized by separate hardware, firmware, or softwarecomponents. Rather, functionality associated with one or more modules orunits may be performed by separate hardware, firmware, or softwarecomponents, or integrated within common or separate hardware, firmware,or software components.

The techniques described in this disclosure may also be embodied orencoded in an article of manufacture including a computer-readablestorage medium encoded with instructions. Instructions embedded orencoded in an article of manufacture including a computer-readablestorage medium encoded, may cause one or more programmable processors,or other processors, to implement one or more of the techniquesdescribed herein, such as when instructions included or encoded in thecomputer-readable storage medium are executed by the one or moreprocessors. Computer readable storage media may include random accessmemory (RAM), read only memory (ROM), programmable read only memory(PROM), erasable programmable read only memory (EPROM), electronicallyerasable programmable read only memory (EEPROM), flash memory, a harddisk, a compact disc ROM (CD-ROM), a floppy disk, a cassette, magneticmedia, optical media, or other computer readable media. In someexamples, an article of manufacture may include one or morecomputer-readable storage media.

In some examples, a computer-readable storage medium may include anon-transitory medium. The term “non-transitory” may indicate that thestorage medium is not embodied in a carrier wave or a propagated signal.In certain examples, a non-transitory storage medium may store data thatcan, over time, change (e.g., in RAM or cache).

Various examples have been described. These and other examples arewithin the scope of the following claims.

1-20. (canceled)
 21. A method comprising: displaying a graphical userinterface including a committed-text region, a text-suggestion region,and a graphical keyboard comprising a plurality of keys; detecting, atregions of a presence-sensitive input device that correspond tolocations of a display device at which at least two different keys fromthe plurality of keys are displayed, a single continuous gesture;determining, by a computing device, based on the single continuousgesture, a candidate phrase comprising at least two candidate words, afirst candidate word from the at least two candidate words comprising afirst character associated with a first key from the at least twodifferent keys and a second candidate word from the at least twocandidate words comprising a second character associated with a secondkey from the at least two different keys; displaying, in thecommitted-text region, a respective character associated with each keyfrom the plurality of keys that is selected by the single continuousgesture; and displaying, in the text-suggestion region, each word in thecandidate phrase separated by a space character.
 22. The method of claim21, wherein the text-suggestion region is divided into threetext-suggestion regions and displaying each word in the candidate phraseseparated by a space character comprises displaying each word in thecandidate phrase separated by a space character in one region of thethree text-suggestion regions.
 23. The method of claim 22, wherein theone region of the three text-suggestion regions is a middle regiondisplayed between the other two regions of the three text-suggestionregions.
 24. The method of claim 21, wherein the candidate phrasecomprises one or more characters that are not associated with any keysfrom the plurality of keys that are selected by the single continuousgesture.
 25. The method of claim 21, the committed-text region, withinthe graphical user interface, is positioned between the committed-textregion and the graphical keyboard.
 26. The method of claim 21, whereinno spacebar key from the plurality of keys is selected by the singlecontinuous gesture.
 27. The method of claim 21, wherein displaying, inthe committed-text region, a respective character associated with eachkey from the plurality of keys that is selected by the single continuousgesture comprises: responsive to detecting a single lift-off event atthe presence-sensitive input device, displaying, in the committed-textregion, the respective character associated with each key from theplurality of keys that is selected by the single continuous gesture. 28.A computing device comprising: a display device; a presence-sensitiveinput device; and at least one processor configured to: display, on thedisplay device, a graphical user interface including a committed-textregion, a text-suggestion region, and a graphical keyboard comprising aplurality of keys; detect, at regions of the presence-sensitive inputdevice that correspond to locations of the display device at which atleast two different keys from the plurality of keys are displayed, asingle continuous gesture; determine, based on the single continuousgesture, a candidate phrase comprising at least two candidate words, afirst candidate word from the at least two candidate words comprising afirst character associated with a first key from the at least twodifferent keys and a second candidate word from the at least twocandidate words comprising a second character associated with a secondkey from the at least two different keys; display, in the committed-textregion, a respective character associated with each key from theplurality of keys that is selected by the single continuous gesture; anddisplay, in the text-suggestion region, each word in the candidatephrase separated by a space character.
 29. The computing device of claim28, wherein the text-suggestion region is divided into threetext-suggestion regions and displaying each word in the candidate phraseseparated by a space character comprises displaying each word in thecandidate phrase separated by a space character in one region of thethree text-suggestion regions.
 30. The computing device of claim 29,wherein the one region of the three text-suggestion regions is a middleregion displayed between the other two regions of the threetext-suggestion regions.
 31. The computing device of claim 28, whereinthe candidate phrase comprises one or more characters that are notassociated with any keys from the plurality of keys that are selected bythe single continuous gesture.
 32. The computing device of claim 28, thecommitted-text region within the graphical user interface, is positionedbetween the committed-text region and the graphical keyboard.
 33. Thecomputing device of claim 28, wherein no spacebar key from the pluralityof keys is selected by the single continuous gesture.
 34. The computingdevice of claim 28, wherein the at least one processor is configured todisplay, in the committed-text region, a respective character associatedwith each key from the plurality of keys that is selected by the singlecontinuous gesture by: responsive to detecting a single lift-off eventat the presence-sensitive input device, displaying, in thecommitted-text region, the respective character associated with each keyfrom the plurality of keys that is selected by the single continuousgesture.
 35. A non-transitory computer-readable storage mediumcomprising instructions that, when executed, cause at least oneprocessor of a computing device to: display, on a display device, agraphical user interface including a committed-text region, atext-suggestion region, and a graphical keyboard comprising a pluralityof keys; detect, at regions of a presence-sensitive input device thatcorrespond to locations of the display device at which at least twodifferent keys from the plurality of keys are displayed, a singlecontinuous gesture; determine, based on the single continuous gesture, acandidate phrase comprising at least two candidate words, a firstcandidate word from the at least two candidate words comprising a firstcharacter associated with a first key from the at least two differentkeys and a second candidate word from the at least two candidate wordscomprising a second character associated with a second key from the atleast two different keys; display, in the committed-text region, arespective character associated with each key from the plurality of keysthat is selected by the single continuous gesture; and display, in thetext-suggestion region, each word in the candidate phrase separated by aspace character.
 36. The non-transitory computer-readable storage mediumof claim 35, wherein the text-suggestion region is divided into threetext-suggestion regions and the at least one processor is configured todisplay each word in the candidate phrase separated by a space characterby displaying each word in the candidate phrase separated by a spacecharacter in one region of the three text-suggestion regions.
 37. Thenon-transitory computer-readable storage medium of claim 36, wherein theone region of the three text-suggestion regions is a middle regiondisplayed between the other two regions of the three text-suggestionregions.
 38. The non-transitory computer-readable storage medium ofclaim 35, wherein the candidate phrase comprises one or more charactersthat are not associated with any keys from the plurality of keys thatare selected by the single continuous gesture.
 39. The non-transitorycomputer-readable storage medium of claim 35, wherein the committed-textregion, within the graphical user interface, is positioned between thecommitted-text region and the graphical keyboard.
 40. The non-transitorycomputer-readable storage medium of claim 35, wherein no spacebar keyfrom the plurality of keys is selected by the single continuous gesture.